Apparatus and methods for intelligent deployment of network infrastructure based on tunneling of ethernet ring protection

ABSTRACT

Apparatus and methods for intelligent deployment and transition from a first network infrastructure to a second network infrastructure. Various embodiments of the present disclosure are directed to, among other things, methods and apparatus that leverage tunneling of Ethernet ring network technologies. In one exemplary embodiment, a modified implementation of the ITU-T G.8032 data link protocol is combined with Multiprotocol Label Switching (MPLS) transport networks to provide Carrier Ethernet and Retail Ethernet services. Unlike existing network infrastructure, the exemplary MPLS network aggregates traffic between the base station (BS) and mobile switching center (MSC) within a logical ring network topology.

COPYRIGHT

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

1. Technological Field

The disclosure relates generally to the field of data and contentdelivery. In one exemplary aspect, the disclosure relates to linkaggregation technologies that enable intelligent deployment andtransition from one network infrastructure to another networkinfrastructure.

2. Description of Related Technology

Within the telecommunications arts, the term “backhaul” refers generallyto the high-speed links between the core network and the sub-networks atthe “edge” of the network. Generally, backhaul services are provided bya commercial wholesale bandwidth provider who guarantees certain Qualityof Service (QOS) or Service Level Agreements (SLAs) to e.g., a retailer(e.g., Internet Service Provider (ISP)), network operator (e.g.,cellular network operator), etc.

As a brief aside, legacy backhaul technologies generally consist ofe.g., Synchronous Optical Networking (SONET) and/or older T-Carriers(e.g., T1, T2, T3). SONET networks transfer multiple digital bit streamsover optical fiber using lasers or highly coherent light fromlight-emitting diodes (LEDs). SONET is based on fixed size “containers”that encapsulate data; the encapsulated data can be further formattedaccording to e.g., traditional telephony, Asynchronous Transfer Mode(ATM), Ethernet, and TCP/IP traffic.

Many Cellular-Tower Backhaul (CTBH) networks are migrating toMultiprotocol Label Switching (MPLS) infrastructure. For example, MPLSrouters can be used in base stations (BS), Mobile Switching Centers(MSC), and each node between. While the MPLS architecture is oftenexpensive, MPLS supports the Service Level Agreements (SLA) that themobile operators require. More directly, commercial wholesale SLAs (alsoreferred to herein as “Carrier Ethernet”) require substantially higherQoS (e.g., enhanced protection mechanisms and timing accuracy) thanso-called “Retail Ethernet” services, which do not have such stringentrequirements.

MPLS is a newer backhaul technology that routes variable length datafrom one network node to another based on path labels, rather thannetwork addresses. In the exemplary implementation, each path labelidentifies virtual links (paths) between distant nodes rather thanendpoint addresses. Packet-forwarding decisions are made solely on thecontents of the label, without the need to examine the packet itself.MPLS routing advantageously avoids complex endpoint address lookups outof a routing table, which significantly reduces overall transport times.

Given the cost of MPLS, it is desirable to couple with anothernetworking technology (e.g., Ethernet) to deliver data to its finaldestination via so-called “access networks”. As with SONET, MPLS canencapsulate data for a wide range of networking technologies includinge.g., Ethernet, Asynchronous Transfer Mode (ATM), Frame Relay, DigitalSubscriber Line (DSL), etc.

Backhaul providers ideally seek to maximize the amount ofrevenue/utilization from large capital expenditures (CAPEX). Sincebackhaul providers have already significantly invested capital inboosting MPLS infrastructure to support Carrier Ethernet, solutions areneeded to flexibly provide both Carrier Ethernet and Retail Ethernetwith maximal network infrastructure reuse.

SUMMARY OF THE DISCLOSURE

The present disclosure provides, inter alia, apparatus and methods forlink aggregation technologies that enable intelligent deployment andtransition from one network infrastructure to another networkinfrastructure.

A method for enabling intelligent deployment and transition from a firstnetwork infrastructure to a second network infrastructure is disclosed.In one embodiment, the method includes: providing a distribution networkcomprising a plurality of nodes configured to route one or more dataframes, where each of said one or more data frames comprises anencapsulated data and a routing label; providing at least one ingressdevice, where said at least one ingress device is configured to generatesaid one or more data frames and transmit said one or more data framesvia said distribution network to said at least one egress device;providing at least one egress device, where said at least one egressdevice is configured to receive said one or more data frames via saiddistribution network; assigning each of said plurality of nodes and saidat least one ingress device and said at least one egress device to aring network; and transacting said one or more data frames via said ringnetwork.

In one variant, said logical ring network comprises at least a firstactive path in a primary ring and a second active path in a secondaryring, where said one or more data frames are transacted via said firstactive path in the primary ring and said second active path in thesecondary ring. Additionally, said logical ring network may furthercomprise at least a third path in a standby primary ring and a fourthpath in a standby secondary ring, where said third path in said standbyprimary ring and said fourth path in said standby secondary ring areblocked. Responsive to detecting a ring failure, the method may includeunblocking at least one of said third path in said standby primary ringand said fourth path in said standby secondary ring and thereaftertransacting data via said unblocked at least one path.

In another variant, said ring network is configured to route said one ormore data frames based on associated labels.

In a third variant, said at least one egress device comprises a cellulartower and said at least one ingress device comprises a mobile servicesprovider (MSP) router.

In a fourth variant, said at least one egress device is configured toprovide retail Internet service and said at least one ingress devicecomprises an internet services provider (ISP) router.

In a fifth variant, said ring network services a combination of Retailand Carrier Ethernet applications characterized by distinct ServiceLevel Agreements (SLAB).

A premises apparatus is disclosed. In one embodiment, the premisesapparatus includes: a first network interface configured to communicatewith a backhaul network comprising at least a ring network; a consumerpremises interface configured to communicate with an edge network; aprocessor; and a non-transitory computer readable medium comprising atleast one computer program. In one exemplary embodiment, the computerprogram is configured to, when executed by said processor, cause saidpremises apparatus to: receive at least one data frame from a firstother node of said ring network; transmit said at least one data frameto a second other node of said ring network; and determine when said atleast one data frame comprises at least one packet for said edgenetwork; and route said at least one packet via said edge network.

In one variant, said first network interface comprises a data link layerinterface. One such implementation may include a Multiprotocol LabelSwitching (MALS) compliant interface, and/or operate within a ringnetwork that includes a ITU-T G.8032 compliant ring network.

In one variant, said edge network comprises a cellular tower site.

A network router apparatus is disclosed. In one embodiment, the networkrouter apparatus includes: a first network interface configured toconnect to a backhaul network comprising at least a ring network; aprocessor; and a non-transitory computer readable medium comprising atleast one computer program. In one exemplary implementation, thecomputer program is configured to, when executed by said processor,cause said network router apparatus to: receive at least one data framefrom a first other node of said ring network, where said at least onedata frame comprises a first label associated with said network routerapparatus and an encapsulated data; replace said first label with asecond label associated with a second other node of said ring network;and transmit said at least one data frame to said second other node ofsaid ring network.

In one variant, said at least one data frame comprises a three (3) labelstack which includes: (i) a first stack layer configured to providerouting information, (ii) a second stack layer configured to specify atransport network service endpoint, and (iii) a third stack layerconfigured to identify an appropriate private network for saidencapsulated data. Each of the first, second, and third stack layers areassociated with corresponding quality of service (QoS) informationuseful for prioritization within the associated layer.

In a second variant, said network router apparatus supports both labelbased routing and network address based routing.

An aggregator apparatus is disclosed. In one embodiment, the aggregatorapparatus includes: a first network interface configured to connect to abackhaul network comprising at least a ring network; a backboneinterface configured to connect to a backbone network; a processor; anda non-transitory computer readable medium comprising at least onecomputer program. In one embodiment, the computer program is configuredto, when executed by said processor, cause said aggregator apparatus to:receive at least one data frame from a first other node of said ringnetwork; transmit said at least one data frame to a second other node ofsaid ring network; and determine when said at least one data framecomprises at least one packet for said backbone network; and route saidat least one packet via said backbone network.

A method of operating a distribution network comprising a plurality ofnodes configured to route one or more data frames comprising anencapsulated data and a routing label is disclosed. The networkincludes: at least one ingress device configured to generate said one ormore data frames and transmit said one or more data frames to said atleast one egress device, the at least one egress device configured toreceive said one or more data frames. In one embodiment, the methodincludes: assigning each of said plurality of nodes and said at leastone ingress device and said at least one egress device to a ringnetwork; and transacting said one or more data frames via said ringnetwork. In one such variant, the ring network is closed over aMultiprotocol Label Switching (MPLS) transport network.

In a further aspect of the disclosure, a method for operating a contentdistribution network is disclosed. In one embodiment, the methodincludes: (i) enabling at least one user interface to communicate withan edge network; (ii) receiving at least one data frame from a firstother node of a backhaul network, the backhaul network including atleast a ITU-T G.8032 compliant logical ring network; (iii) transmittingthe at least one data frame to a second other node of the ITU-T G.8032compliant logical ring network; (iv) determining when the at least onedata frame comprises at least one packet for the edge network; and (v)routing the at least one packet via the edge network.

In one variant, the at least one user interface includes a MultiprotocolLabel Switching (MPLS) compliant interface. In another variant, the edgenetwork includes a cellular tower site.

In a further aspect of the disclosure, a non-transitorycomputer-readable apparatus is disclosed. In one embodiment, thenon-transitory computer-readable apparatus includes media implemented tostore a computer program thereon. In one variant, the computer programincludes a plurality of instructions which are implemented to, whenexecuted by a processor of a network apparatus: (i) receive via aninterface of the network apparatus at least one data frame originatedfrom a first node of a backhaul logical ring network; (ii) insert intothe data frame a second label associated with a second node of thelogical ring network in place of a first label associated with a networkrouter; (iii) transmit the at least one data frame to a second node ofthe backhaul logical ring network; (iv) determine that the at least onedata frame comprises at least one packet for a cellular site associatedwith an edge network; and (v) route at least the at least one packet viathe edge network to the cellular site. In one implementation, thelogical ring network is ITU-T G.8032 compliant.

In yet a further aspect of the disclosure, an apparatus is disclosed. Inone embodiment, the apparatus includes: (i) a first network interfaceimplemented to communicate with a backhaul network comprising at least alogical ring network; (ii) an interface configured to communicate withan edge network; (iii) a processor; and (iv) a non-transitory computerreadable medium comprising at least one computer program.

In one variant, the at least one computer program is implemented to,when executed by the processor, cause the apparatus to: (i) receive atleast one data frame from a first other node of the logical ringnetwork; (ii) transmit the at least one data frame to a second othernode of the logical ring network; (iii) determine when the at least onedata frame comprises at least one packet for a cellular site associatedwith the edge network; and (iv) route the at least one packet via theedge network to the cellular site. In one implementation, the firstnetwork interface includes a Multiprotocol Label Switching (MPLS)compliant interface. In another implementation, the logical ring networkis ITU-T G.8032 compliant.

These and other aspects of the disclosure shall become apparent whenconsidered in light of the disclosure provided herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a graphical representation of one exemplary embodiment of aCellular-Tower Backhaul (CTBH) network migration from Phase I to PhaseII, illustrating a staged deployment of capital equipment, in accordancewith various aspects of the present disclosure.

FIG. 1B is a graphical representation of one exemplary embodiment of aCTBH network migration from Phase II to Phase III, illustrating atransition from one network technology to another network technology, inaccordance with various aspects of the present disclosure.

FIG. 2 is a detailed logical block diagram of an exemplary Phase II CTBHarchitecture, in accordance with various aspects of the presentdisclosure.

FIG. 3A is a detailed logical block diagram of an exemplary Phase IIICTBH architecture, in accordance with various aspects of the presentdisclosure.

FIG. 3B is a detailed logical block diagram of an exemplaryheterogeneous Phase II/Phase III CTBH architecture, in accordance withvarious aspects of the present disclosure.

FIG. 4 is a logical flow diagram of one embodiment of a generalizedmethod for intelligent deployment and transition from a first networkinfrastructure to a second network infrastructure, in accordance withvarious aspects of the present disclosure.

FIG. 5 is a logical block diagram of an exemplary embodiment of aConsumer Premises Equipment (CPE) configured to provide networkedoperation in conjunction with the generalized architecture of FIGS. 3Aand 3B.

FIG. 6 is a logical block diagram of an exemplary embodiment of a Layer2 Aggregator device configured to provide networked operation inconjunction with the generalized method architecture of FIGS. 3A and 3B.

FIG. 7 is a logical block diagram of an exemplary embodiment of a Layer2 Network Interface Device configured to provide networked operation inconjunction with the generalized architecture of FIGS. 3A and 3B.

FIG. 8 is a logical block diagram of one exemplary embodiment of amethod for implementing an ITU-T G.8032 ring network within a backhauldistribution network with MPLS capability, in accordance with variousaspects of the present disclosure.

All Figures © Copyright 2012-2013 Time Warner Cable, Inc. All rightsreserved.

DETAILED DESCRIPTION

Reference is now made to the drawings wherein like numerals refer tolike parts throughout.

As used herein, the terms “client device” and “end user device” include,but are not limited to, set-top boxes (e.g., DSTBs), gateways, modems,personal computers (PCs), and minicomputers, whether desktop, laptop, orotherwise, and mobile devices such as handheld computers, PDAs, personalmedia devices (PMDs), tablets, and smartphones.

As used herein, the term “computer program” or “software” is meant toinclude any sequence or human or machine cognizable steps which performa function. Such program may be rendered in virtually any programminglanguage or environment including, for example, C/C++, FORTRAN, COBOL,PASCAL, assembly language, markup languages (e.g., HTML, SGML, XML,VoXML), and the like, as well as object-oriented environments such asthe Common Object Request Broker Architecture (CORBA), Java™ (includingJ2ME, Java Beans, etc.), Binary Runtime Environment (e.g., BREW), andthe like.

The terms “Consumer Premises Equipment (CPE)” and “host device” referwithout limitation to any type of electronic equipment located within aconsumer's or user's premises and connected to a network. The term “hostdevice” includes terminal devices that have access to digital televisioncontent via a satellite, cable, or terrestrial network. The host devicefunctionality may be integrated into a digital television (DTV) set. Theterm “consumer premises equipment” (CPE) includes such electronicequipment such as set-top boxes, televisions, Digital Video Recorders(DVR), gateway storage devices (FURNACE), and ITV Personal Computers.

As used herein, the term “DOCSIS” refers to any of the existing orplanned variants of the Data Over Cable Services InterfaceSpecification, including for example DOCSIS versions 1.0, 1.1, 2.0 and3.0.

As used herein, the term “headend” refers generally to a networkedsystem controlled by an operator (e.g., an MSO or multiple systemsoperator) that distributes programming to MSO clientele using clientdevices. Such programming may include literally any informationsource/receiver including, inter alia, free-to-air TV channels, pay TVchannels, interactive TV, and the Internet.

As used herein, the terms “Internet” and “internet” are usedinterchangeably to refer to inter-networks including, withoutlimitation, the Internet.

As used herein, the terms “microprocessor” and “digital processor” aremeant generally to include all types of digital processing devicesincluding, without limitation, digital signal processors (DSPs), reducedinstruction set computers (RISC), general-purpose (CISC) processors,microprocessors, gate arrays (e.g., FPGAs), PLDs, reconfigurablecomputer fabrics (RCFs), array processors, secure microprocessors, andapplication-specific integrated circuits (ASICs). Such digitalprocessors may be contained on a single unitary IC die, or distributedacross multiple components.

As used herein, the terms “MSO” or “multiple systems operator” referwithout limitation to a cable, fiber to the home (FTTH), fiber to thecurb (FTTC), satellite, Hybrid Fiber Copper (HFCu), or terrestrialnetwork provider having infrastructure required to deliver servicesincluding programming and data over those mediums.

As used herein, the terms “network” and “bearer network” refer generallyto any type of telecommunications or data network including, withoutlimitation, hybrid fiber coax (HFC) networks, HFCu networks, satellitenetworks, telco networks, and data networks (including MANs, WANs, LANs,WLANs, internets, and intranets). Such networks or portions thereof mayutilize any one or more different topologies (e.g., ring, bus, star,loop, etc.), transmission media (e.g., wired/RF cable, RF wireless,millimeter wave, optical, etc.) and/or communications or networkingprotocols.

As used herein, the term “network interface” refers to any signal, data,or software interface with a component, network or process including,without limitation, those of the FIREWIRE (e.g., FW400, FW800, etc.),USB (e.g., USB2), Ethernet (e.g., 10/100, 10/100/1000 (GigabitEthernet), 10-Gig-E, etc.), MOCA, Coaxsys (e.g., TVnet™), radiofrequency tuner (e.g., in-band or OOB, cable modem, etc.), WI-FI(802.11), WIMAX (802.16), PAN (e.g., 802.15), or IRDA families.

As used herein, the term “node” refers to any functional entityassociated with a network, such as for example: CPE, server, gateway,router, Optical Line Terminal (OLT), Optical Network Unit (ONU), etc.whether physically discrete or distributed across multiple locations.

As used herein, the term “server” refers to any computerized component,system or entity regardless of form which is adapted to provide data,files, applications, content, or other services to one or more otherdevices or entities on a computer network.

As used herein, the term “wireless” means any wireless signal, data,communication, or other interface including without limitation WI-FI,BLUETOOTH, 3G (3GPP/3GPP2), HSDPA/HSUPA, TDMA, CDMA (e.g., IS-95A,WCDMA, etc.), FUSS, DSSS, GSM, PAN/802.15, WIMAX (802.16), 802.20,narrowband/FDMA, OFDM, PCS/DCS, LTE/LTE-A, analog cellular, CDPD,satellite systems, millimeter wave or microwave systems, acoustic, andinfrared (i.e., IrDA).

Overview

Various embodiments of the present disclosure are directed to methodsand apparatus that leverage existing data transfer protocols betweenadjacent nodes in a network (generally referred to as “Layer 2” or the“Data Link Layer”) to provide tunneling of e.g., Ethernet ringprotection. As described in greater detail hereinafter, the exemplarysolutions described herein provide comparable “access network”protection to existing technologies (e.g., MPLS), consistency inoperational support models, and significantly reduced costs for backhaulproviders. More directly, the principles disclosed herein advantageouslyoffer one or more of (i) consistent access architecture for allcommercial Ethernet services (e.g., Carrier Ethernet and RetailEthernet), (ii) reduced operational expenditure (OPEX) (as compared tosupporting distinct Carrier Ethernet and Retail Ethernetinfrastructures), (iii) reduced capital expenditures (CAPEX) (ascompared to the existing access devices used for Carrier Ethernetapplications), and/or (iv) consistent Performance Monitoring (PM) andService Activation Testing (SAT) solutions for all commercial Ethernetservices (both Retail and Wholesale Carrier Ethernet installations).

In one exemplary embodiment of the present disclosure, a modifiedimplementation of the ITU-T G.8032 data link protocol is tunneled viaMultiprotocol Label Switching (MPLS) transport networks to provide alogical Ethernet “ring” network between each Consumer Premises Equipment(CPE) and two “Layer 2” (L2) Aggregator Devices. Unlike existing networkinfrastructure, the exemplary ring network tunnels traffic between atleast an ingress point or node (such as a base station (BS)) and atleast an egress point or node (e.g., a mobile switching center (MSC)) toform a single logical ring network topology that spans a distributionnetwork infrastructure. More directly, rather than closing the networkring topology at the “access network” (as is done in existing ringnetworks), the ring is closed at the ingress and egress nodes (such as aBS and MSC), and traffic is logically tunneled via an interveningdistribution network that connects the ingress and egress nodes. In thismanner, traffic can be transferred through the nodes of the ring networkas if the nodes were directly connected to one another, regardless ofthe operation of the intervening distribution network (e.g., a MPLStransport network).

The exemplary embodiments of the network infrastructure set forth hereinadvantageously do not require significant new investment in capitalequipment, and can be deployed in an incremental manner. Additionally,each node of the exemplary distribution network only needs to supportthe tunneling protocol (e.g., data link protocol capabilities (Layer 2capabilities)) whereas existing networks require each intermediary nodeto perform full network address resolution (Layer 3 capabilities). Theselower capabilities requirements directly translate to less expensiveequipment that can be used and/or deployed.

Moreover, existing vendor products typically cater to either CarrierEthernet or Retail Ethernet, and hence are frequently notinterchangeable. While all single vendors ensure that their products arecompatible with their own product offerings, vendor cross-compatibilityis not assured, and such issues may complicate e.g., performancemonitoring, SLA compliance, etc. The ability to source interchangeablecomponents from multiple different vendors ensures market competitionand promotes technical innovation. The various disclosed embodiments donot require specialized functionality, and can be easily handled bycommodity components, thus greatly reducing or even eliminatingcross-compatibility issues and enabling significant multi-sourceopportunities.

Timing synchronization is often a critical requirement for most CarrierEthernet applications (e.g., cellular network deployments, etc.).Similarly, from a network management perspective, scalable solutions foradding additional bandwidth are highly desirable. As described ingreater detail herein, the disclosed embodiments further greatlysimplify both timing synchronization and scalable network deployment.Still other synergies described in greater detail hereinafter will bemade apparent to those skilled in the art upon reading and understandingthe present specification.

Detailed Description of Exemplary Embodiments

Exemplary embodiments of the apparatus and methods of the presentdisclosure are now described in detail. While these exemplaryembodiments are described in the context of the aforementionedCellular-Tower Backhaul (CTBH) network system architecture, the generalprinciples and advantages of the disclosure may be extended to othertypes of networks and architectures, whether implemented within the corenetwork, backhaul, edge networks, etc., the following therefore beingmerely exemplary in nature.

It will also be appreciated that while described generally in thecontext of a commercial wholesale bandwidth provider with CarrierEthernet and Retail Ethernet capabilities, the present disclosure may bereadily adapted to other types of environments (e.g.,commercial/enterprise, government/military, etc.) as well. Myriad otherapplications are possible.

While the terms “ingress” and “egress” are used with reference to thevarious functions of the nodes described herein, it should beappreciated that such usage is provided solely for clarity. In fact, itis readily appreciated that typical nodes, applications, and/ortransactions are bidirectional in nature, and thus nodes may possessboth ingress and egress capabilities.

Other features and advantages of the present disclosure will immediatelybe recognized by persons of ordinary skill in the art with reference tothe attached drawings and detailed description of exemplary embodimentsas given below.

Exemplary Network Architecture—

Referring now to FIGS. 1A and 1B, one exemplary configuration of aCellular-Tower Backhaul (CTBH) network migration 100 is illustrated. TheCTBH network comprises: a mobile switching center (MSC) 102 (which linksto the Mobile Service Provider (MSP)), a primary hub 104, a secondaryhub 106, and a cellular site (base station (BS) deployment) 108. Asshown, each phase of the CTBH network migration, replaces existingnetwork infrastructure components to keep pace with e.g., technologylimitations, new requirements and solution enhancements, etc.

Referring now to the Phase I Architecture of FIG. 1A, the backhaul isprovided via Synchronous Optical Networking (SONET)/Synchronous DigitalHierarchy (SDH) components which do not support Multiprotocol LabelSwitching (MPLS) functionality.

FIG. 1A further depicts the transition from Phase I to Phase II beingperformed by successively replacing legacy SONET/SDH components withIP/MPLS capable components. Phase II is characterized by full IP/MPLSconnectivity from the MSC to cellular site. Phase II deploymentsmaintain a separation between Wholesale Ethernet and Retail Ethernet,due to the differences in SLA requirements and access technologies. Itshould be noted that, multi-homed cell sites (a cellular networkoperator requirement) require IP/MPLS Layer 3 CPE deployments in PhaseII where multi-home capabilities were not available with Ethernet. InPhase III (described hereinafter), the cell site can directly transitionto IP/MPLS Layer 2 Ethernet CPE deployments with ITU-T G.8032 supportfor Ethernet rings.

Referring now to FIG. 2, a more detailed representation of an exemplaryconfiguration of Phase II CTBH architecture is illustrated for clarity.The cellular tower 202A is connected via physical Ethernet (e.g., agateway) to the components of the distribution hubs (primary andsecondary hubs) 204 and the mobile operator switch (MSC) 206 (e.g., viaa gateway). Logically, the network consists of four (4) bidirectionalEthernet Virtual Circuits (EVC) (or virtual local area networks(VLANs)): a primary active EVC, and a secondary active EVC, a primarystandby EVC, and a secondary standby EVC. In some implementations, therouters may additionally be functionally categorized as a Label SwitchRouters (LSR) that are configured to add a tunnel label to forwardtraffic through a MPLS distribution network, or Label Edge Routers (LER)that are configured to add a service label that is configured to directtraffic to the appropriate customer interface.

As a brief aside, due to various contractual requirements for servicee.g., guaranteed data rates, etc. existing traffic generally providesmultiple hierarchical layers of service redundancy. For example, as usedherein, the terms “active” and “standby” refer to the distributionnetwork operator's physical redundancy circuits. During normal operationthe active circuits transact data; however, in the event of adistribution network failure, the affected data traffic is switched overto their respective standby circuits. Similarly, the terms “primary” and“secondary” refer to logical EVCs provided by the Carrier Ethernetreseller (e.g., Wholesale and/or Retail network providers). Thecommercial Ethernet reseller may provide network protection via e.g.,Bidirectional Forwarding Detection (BFD), to determine when a networkfault requires a switch from the primary EVC to the secondary EVC. Itshould be appreciated that more complex/robust schemes may incorporateadditional levels of backup e.g., tertiary, quaternary, etc.

Each EVC is characterized by an exemplary Label Distribution Protocol(LDP) which transfers label mapping information necessary for MPLSrouting. In the exemplary configuration, two routers with an establishedsession (called “LDP peers”) are provided, and the exchange ofinformation is bidirectional. LDP is used to build and maintain LabelSwitched Path (LSP) databases that are used to forward traffic throughMPLS networks. Each path is configured based on criteria in theforwarding equivalence class (FEC).

A path begins in the illustrated case at a label edge router (LER) or“ingress router”, which makes a decision on which label to prefix to apacket based on the appropriate FEC. The last router in an LSP is calledan “egress router”. Routers in between the ingress and egress routersare herein referred to as “transit routers” or “label switch routers(LSRs)”. Each router forwards the packet along to the next router, whichforwards it to the next router, etc. The penultimate router (second tolast router in the path) removes the outer label and the last router inthe path (i.e., the egress router) removes the inner label from thepacket and forwards the resulting frame based on an appropriate networkprotocol (for example Ethernet for a Carrier Ethernet network). Sincethe LSP transactions are performed at the Data Link Layer (Layer 2) andare transparent to networking protocols (Layer 3), an LSP is alsosometimes referred to as an MPLS tunnel and/or “pseudo-wire”.

As used herein, “Layer 1” (or the “Physical” Layer) refers withoutlimitation to the hardware transmission technology of the network.“Layer 2” (or the “Data Link” Layer) refers without limitation to aprotocol layer that transfers data between adjacent network nodes in awide area network or between nodes on the same local area networksegment. Layer 2 functions include among other things: media accesscontrol (MAC addressing), flow control and error checking. “Layer 3” (orthe “Data Link” Layer) refers to a protocol layer that routes datathroughout the network based on network address resolution, etc. Layer 3functions include among other things: route determination, and packetforwarding, etc. It should be noted that according to the foregoingdescriptions, Ethernet frames constitute a Layer 2 data structure. EachEthernet frame includes e.g., a preamble, a frame delimiter, a MACdestination address, a MAC source address, a data payload, and a framecheck sequence. The Ethernet frame payload typically contains e.g.,TCP/IP addressing, but could be used to encapsulate other protocols.

Referring back to FIG. 2, each cellular tower 202A is connected to acellular gateway 202B which is connected to two (2)geographically-redundant hub site routers (204A, 204B) via Ethernetpoint-to-point links. Typically, the four (4) EVC (primary active EVC,primary standby EVC, secondary active EVC, secondary standby EVC) areconfigured using the remote system IP of each of the MSC routers (204E,204F). In some implementations, an Interior Gateway Protocol (IGP) maybe used to determine the best path. Two (2) LSPs are configured for eachtower virtual local area network (VLAN) and each LSP provides analternate path to one of the two (2) redundant MSC gateway routers(204E, 204F). As shown, the primary and secondary pseudo-wires arestitched to the core transit service at the ingress router. Theresulting configuration provides a primary VLAN that is staticallyconfigured to use the LSP, traversing a first path (from 204A to 204E)that is configured to perform fail-over switching to a secondary LSPtraversing a second path (from 204B to 204E).

Similarly, the secondary VLAN is statically configured to use the LSPtraversing a first port (from 204B to 204F) that is configured toperform fail-over switching to a secondary LSP traversing a second port(from 204A to 204F).

In Phase II, the distribution hubs of the exemplary configurationutilize the Resource Reservation Protocol (RSVP) that enables eitherhosts or routers to request or deliver specific levels of quality ofservice (QoS) for application data streams or flows. RSVP defines howsoftware applications request reservations and relinquish the reservedresources. Typical RSVP operation requires a defined resource allocationreserved in each node along the LSP.

Referring back to FIG. 1B, from Phase II, the transition to Phase IIIresults in the MSC 102 and the cellular sites 108 converting to Layer 2type devices that support ITU-T G.8032 (e.g., replacing IP/MPLS Layer 3CPEs with IP/MPLS Layer 2 CPEs, etc.) and the CTBH is converted to aring network, where the ring is “closed” at the MSC and cellular sites(i.e., the ring spans each node of the CTBH). From Phase III on, theCTBH networks (Carrier Ethernet) have the same infrastructure technologyas Retail Ethernet; thus a common Ethernet infrastructure can supportoperational models for both Carrier Ethernet and Retail Ethernet.

Referring now to FIG. 3A, a more detailed representation of theexemplary Phase III CTBH architecture is illustrated for clarity. Asshown, the ingress and egress routers have been replaced with Data LinkLayer (Layer 2) equipment. Additionally, as shown the Layer 2 routers ateach end of the service are configured to “close” an ITU-T G.8032 ringfor each EVC (VLAN). In one embodiment, at least a portion of the ITU-TG.8032 rings (e.g., the standby circuits) are “blocked” at the Layer-2CPE devices by default (at the cellular tower site) to prevent a network“loop”; the primary active EVC is handled over a first network interfaceand the secondary active EVC is handled on a second network interface.Once a fault has been detected (e.g., in either the primary active EVCor secondary active EVC), the “blocked” ITU-T G.8032 ring (e.g., thestandby circuit) is unblocked, to recover connectivity for the affectedEVC.

Unlike the Phase II CTBH architecture of FIG. 2, the Phase III CTBHarchitecture of FIG. 3A implements a data link layer logical ringnetwork with the Layer 2 CPE 308, and Layer 2 Aggregators 310E, 310F.Specifically, the logical ring network is tunneled from the Layer 2 CPE308, through the distribution hubs 304, to the Layer 2 Aggregators 310E,310F. For example, the primary active EVC consists of the link from theLayer 2 CPE 308, to the distribution hubs (from 304A to 304C) to a firstLayer 2 Aggregator 310E. The primary standby EVC is then used to closethe ring. The primary standby EVC consists of the link from the Layer 2CPE 308, to the distribution hubs (from 304B to 304D) to the secondLayer 2 Aggregator 310F, then to the first Layer 2 Aggregator 310E. Boththe primary active EVC and primary standby EVC have the same ingress andegress points (Layer 2 CPE 308 and Layer 2 Aggregator 310E). Thesecondary active EVC and secondary standby EVC have similar routingbetween the Layer 2 CPE 308 and the Layer 2 Aggregator 310F.

Moreover, in Phase III, the distribution hubs utilize the LDP over RSVP(LDPoRSVP) also known as “tunnel-in-tunnel”. Unlike RSVP, LDPoRSVPutilizes a three (3) label stack which includes: (i) an RSVP labelconfigured to provide RSVP Fast Re-Routing (FRR), (ii) a LDP labelconfigured to provide MPLS end-to-end services over hierarchicalnetworks, and (iii) a Virtual Private Network (VPN) label configured toidentify the appropriate VLAN network for the data. Specifically, theRSVP label is processed by each MPLS router node within the network asthe packet traverses the network, The LDP label is processed by theProvider Edge (PE) Routers (e.g., the ingress and egress routers), andthe VPN label assures the final VLAN destination. Using LDPoRSVPautomates the manual process of stitching pseudo-wires together at IGParea boundaries, while ensuring sub-50 ms restoration of pseudo-wireswithin each MPLS transport network defined by an IGP area or level. Inone such case, the IGP protocol operates according to the Open ShortestPath First (OSPF) protocol. OPSF gathers link state information fromavailable routers and constructs a topology map of the network. Thetopology determines the routing table presented to the Internet Layerwhich makes routing decisions based on the destination IP address foundin IP packets, plus any additional information the routing protocol usedwithin an network domain (area, level, etc.) may consider (i.e. cost,bandwidth, delay, load, reliability, maximum transmission unit (MTU),etc.).

The multi-tiered label structure of LDPoRSVP provides multiple benefitsover LDP. For example, LDPoRSVP enables Fast Re-Routing (FRR) inmulti-area topologies, and dynamic creation of EVCs between serviceendpoints that may reside in different network areas or levels. Moregenerally, LDPoRSVP enables network convergence for Retail andcommercial Carrier Ethernet services, and provides a framework forfuture centralization of all commercial Ethernet services in the future(e.g., Cellular-Tower Backhaul (CTBH), MetroE (Metro Ethernet), EthernetEverywhere and EPON (Ethernet Passive Optical Network)/DPOE (DOCSISProvisioning over Ethernet), etc.). Specifically, LDPoRSVP supportshierarchical networks which rely on LDP to dynamically stitch togetherpseudo-wires that pass from one distinct Interior Gate Protocol (IGP)network domain to another. RSVP is then used within each domain toensure fast re-routing of the portion of the pseudo-wire within thatdomain (area/level). More directly, each IGP domain manages its interiorrouting information between network components (e.g., gateways, routers,etc.) within an Autonomous System (AS) (for example, a collection ofnetworks that belong to the same company). The LDP label provides MPLSrouters with appropriate routing information to create an end-to-endservice that traverses more than a single AS. As used within the relatedarts, an AS is a collection of connected IP routing prefixes under thecontrol of a single administrative authority that presents a common,clearly defined routing policy to the Internet.

Ring topologies are typically very efficient under heavy network loads,do not require significant routing intelligence, and can be quicklyinstalled, reconfigured, and repaired. A ring network topology ischaracterized in that each node logically connects to exactly two (2)other nodes, forming a single continuous pathway for data through eachnode (a ring). Data travels from node to node, with each node betweenhandling every packet. Some variants of ring topologies may overlaymultiple logical rings over a physical connection; or alternatively, mayimplement a ring topology within physical topologies. As should beclear, within the context of the present disclosure, the Layer 2 CPE 308and Layer 2 Aggregators 310E, 310F are physically located at oppositeends of the distribution network, however by using the tunnelingcapabilities of distribution network (e.g., MPLS), the nodes arelogically directly connected to form a “ring”. As used herein, the term“tunnel” refers to the computer networking technique of embedding afirst network protocol within the payload of a second network protocol,so as to logically connect two distinct nodes of the same network(operating with the first network protocol) via a connecting network.Tunneling enables e.g., delivery via mixed network technologies,delivery of secure data via unsecure networks, etc.

Unfortunately, a single failure in a ring network can disrupt the entirenetwork, thus ring networks commonly implement multiple levels ofredundancy. For example, a ring may be rerouted around a failed node,etc. In one particular instance, ITU-T G.8032 (also referred to asEthernet Ring Protection Switching (ERPS)) offers sub-50 ms protectionand recovery switching for Ethernet traffic for ring topologies. As usedherein, ITU-T G.8032 refers to “SERIES G: TRANSMISSION SYSTEMS ANDMEDIA, DIGITAL SYSTEMS AND NETWORKS; Packet over Transportaspects—Ethernet over Transport aspects; SERIES Y: GLOBAL INFORMATIONINFRASTRUCTURE, INTERNET PROTOCOL ASPECTS AND NEXT-GENERATION NETWORKS;Internet protocol aspects—Transport; Ethernet ring protectionswitching”, published February 2012 and incorporated herein by referencein its entirety, which describes protection switching mechanisms, loopprevention, and communication protocols for ring networks. ITU-T G.8032specifically defines: (i) loop avoidance mechanisms, and (ii) learning,forwarding, and Filtering Database (FDB) mechanisms defined in theEthernet flow forwarding function (ETH_FF).

As described within the aforementioned standard, ITU-T G.8032 mandatesthat data traffic may flow on all but one of the ring links. Theunencumbered link is called the Ring Protection Link (RPL), and undernormal conditions the RPL link is blocked, i.e. not used for servicetraffic. One designated Ethernet Ring Node (the RPL Owner Node), isresponsible for blocking traffic at one end of the RPL. Under anEthernet ring failure condition, the RPL Owner Node unblocks its end ofthe RPL (unless the RPL has failed) allowing the RPL to be used fortraffic. The ring failure triggers protection switching of the trafficonto the RPL. Extant ITU-T G.8032 implementations are required (andable) to switchover within 50 ms of a failure event.

Within the context of FIG. 3A, the primary standby EVC is blocked by theLayer 2 CPE 308 during normal operation, but during failover the primarystandby EVC connects the Layer 2 CPE 308, to the distribution hubs (from304B to 304D) to the second Layer 2 Aggregator 310F, then to the secondLayer 2 Aggregator 310E. Similarly, the secondary standby EVC is blockedduring normal operation, but during failover connects the Layer 2 CPE308, to the distribution hubs (from 304A to 304C) to a first Layer 2Aggregator 310E, then to a second Layer 2 Aggregator 310F. Most notably,because the Layer 2 CPE 308 and Layer 2 Aggregator devices 310E, 310Fare operating as an ITU-T G.8032 ring network, the sub-50 ms protectionand recovery switching for Ethernet traffic is supported through thedistribution network. In other words, unlike prior art solutions whichimplement ITU-T G.8032 only within the access network and rely on MPLSprotection in the distribution network, the present disclosure enablesITU-T G.8032 protection from end-to-end. This is a significantimprovement over the existing MPLS protection schemes which are based onBidirectional Forwarding Detection (BED) which can take several hundredmilliseconds to detect and resolve network faults. In some variants, theLayer 2 CPE 308 serves as the Ring Protection Link (RPL) owner toincrease ITU-T G.8032 scalability by distributing ring managementfunctionality. Alternatively, one of the Layer 2 Aggregator devices310E, 310F may be an RPL owner. Furthermore it is appreciated that sinceeach of the ring networks is logically distinct, various hybridarrangements may be implemented (where some ring networks are managed bythe Layer 2 CPE 308, and others are managed by the Layer 2 Aggregatordevices 310E, 310F) based on e.g., device capability, networkcapability, contractual requirements, etc.

Additionally, it should be noted that the Phase III CTBH architectureretains the MPLS core components from Phase II CTBH architecture whichsignificantly reduces the overall cost of migration (CAPEX). The PhaseIII CTBH only requires the MPLS routers to forward frames (thedistribution infrastructure does not interpret, manipulate or otherwiseaffect the contents of the frame). Consequently, the exemplary Phase IIICTBH is not susceptible to multi-vendor interoperability issues fore.g., the aforementioned ITU-T G.8032. In some cases, the MSC Layer 2Aggregators 310E, 310F may be paired with the same vendor's Layer 2 CPE308 (the intervening distribution infrastructure may be commoditycomponents) to simplify the systems integration efforts associated withFault, Configuration, Accounting, Performance and Security (FCAPS)Management functions that would otherwise be complicated by disparatevendor equipment at each end of a service.

Still further, those of ordinary skill in the related arts will readilyappreciate that the tunneled link between the nodes of the ring networkprovides multiple additional benefits. Each tunnel operates as a directlogical connection (e.g., without higher level network routing, and/orunpredictable delays). Thus, the two ends of the tunneled link cansupport timing constraints which may otherwise be untenable. Forexample, a Layer 2 CPE device (such as a cellular tower) with a tunnelto a Layer 2 Aggregator of the Core Network can transact time sensitivemessaging (such as IEEE 1588 synchronization messages which are requiredfor Carrier Ethernet installations) for CTBH applications (LTE,LTE-Advanced, 4G etc.)

Similarly, from a network management perspective, by providing eachLayer 2 CPE device with its own tunneled logical ring, the nodes of eachring can individually monitor performance, activate/deactivate service,and/or test capabilities without affecting the other logical rings. Forexample, a first ring network may consist of: a first Layer 2 CPE, afirst Layer 2 Aggregator, and a second Layer 2 Aggregator; a second ringnetwork may consist of: a second Layer 2 CPE, the same first Layer 2Aggregator, and the same second Layer 2 Aggregator; a third ring networkmay consist of a third Layer 2 CPE, and the same Layer 2 Aggregators.Each of the first second and third ring networks are individuallytunneled. Where, the first Layer 2 CPE 308 is the RPL owner, the Layer 2CPE 308 can activate/deactivate its ring network (the first ringnetwork) without affecting the other Layer 2 CPEs. Similarly, where aLayer 2 Aggregator device 310E, 310F is the RPL owner, Layer 2Aggregator device can individually activate/deactivate each ringnetworks associated with a Layer 2 CPE 308 without affecting the otherLayer 2 CPEs. Such functionality enables layered network re-convergence(i.e., consolidation of network infrastructure technologies) in that theaddition (or removal) of a Layer 2 CPE can be handled without disruptingexisting networks.

Finally, as shown, each Layer 2 Aggregator 310E, 310F provides anaggregated (or trunked) data link to the MSC, and each Layer 2 CPE 308provides a data link to the cell tower 302. The trunked bandwidth issized sufficiently to accommodate the total number of Layer 2 CPEs. Forexample, a 10 Gb/s data trunk can support: up to one hundred (100) Layer2 CPEs with 100 Mb/s links; up to fifty (50) Layer 2 CPEs with 200 Mb/slinks, etc. While the foregoing data links are undifferentiated betweenLayer 2 CPEs, it is appreciated that a Layer 2 Aggregator is in no wayso limited, and may freely aggregate data links of different bandwidthsfrom different Layer 2 CPEs. Still other implementations of the Layer 2Aggregators may reserve a first portion of bandwidth for legacyoperation, and a second portion of bandwidth for operation in accordancewith the various principles described herein (e.g., tunneled Ethernetring networks). Similarly, Layer 2 Aggregators may maintain Class ofService (CoS) requirements independently for each EVC. Commonimplementations of CoS include e.g., standard queuing mechanisms basedon a priority field of an IEEE 802.1Q tag (included within an Ethernetframe) and/or the so-called MPLS experimental (EXP) field (which iscommonly used to ensure CoS through the transport or network).

As shown in FIG. 3B, a representation of a combination of Phase II CTBHand Phase III CTBH architecture is provided to demonstrateinteroperability. The first cell site 352A communicates with associateddistribution hubs 354A, 354B and hybrid Layer 2/3 Aggregators 360 vialegacy Ethernet point-to-point links. In parallel, the second cell site352B operates via the aforementioned tunneled Ethernet ring networkoperation in communication with the Layer 2 CPE 358 and hybrid Layer 2/3Aggregators 360. The hybrid Layer 2/3 Aggregators combine the data linksinto a trunked data link for the MSC, allowing both the MPLS and IEEE802.1Q frames to traverse the same Ethernet links between the MSC Layer2 Aggregators 360 and the penultimate routers 354C, 354D.

Methods—

Referring now to FIG. 4, one generalized method 400 for intelligentdeployment and transition from a first network infrastructure to asecond network infrastructure.

At step 402 of the method 400, data link capable network equipment isdeployed. Common examples of network equipment include e.g., routers,switches, Layer 2 Consumer Premises Equipment (CPE), Multiprotocol LabelSwitching (MPLS) transport routers, Network Interface Device (NID), andLayer 2 Aggregator devices etc.

For example, in one exemplary embodiment, the following network entitiesare deployed, one or more Layer 2 CPE (coupled to e.g., a cellulartower), a plurality of MPLS network routers, and one or more Layer 2Aggregators (coupled to e.g., a Mobile Services Provider (MSP)). TheLayer 2 CPE and Layer 2 Aggregators connect to the ingress and/or egresspoints for a “tunnel”; where the tunnel connects two (2) distinctportions of a logical network. These logical tunnels enable theaforementioned backhaul capabilities (high speed transfers of largeamounts of data between the one or more Layer 2 CPE and the one or moreLayer 2 Aggregators).

Those of ordinary skill in the related arts will readily appreciate thatthe presented deployment is merely illustrative, and in no way limitsthe myriad of network deployments that are possible given the contentsof the present disclosure. Moreover it should be appreciated that“staged” deployments are commonly used in practical implementations(e.g., where budgetary considerations preclude massive capital and/oroperational investments). For example, in one such deployment scheme, afirst deployment stage may include installing a plurality of Layer 2capable network routers. At a later point, a second deployment stage mayinclude installation of Layer 2 Aggregators. Subsequently thereafter, athird deployment stage may include installation of Layer 2 capable CPE.

At step 404 of the method 400, the deployed network equipment isconfigured to operate according to one or more ring network topologies.Each ring network minimally comprises three (3) network nodes which arearranged such that each network node connects to exactly two (2) othernodes. The ring network is configured so as to support a singlecontinuous pathway for signals through each node (i.e., a “ring”). Inone exemplary embodiment, each ring network is characterized by a pathfrom a Layer 2 CPE, to a first Layer 2 Aggregator, to a second Layer 2Aggregator, back to the Layer 2 CPE. Those of ordinary skill in therelated arts will readily appreciate that other network topologies(which conform to ring network constraints) are equally suitable (e.g.,two (2) Layer 2 CPEs and two (2) Layer 2 Aggregators, three (3) Layer 2CPEs and two (2) Aggregators, etc.).

Additionally, exemplary implementations may further augment the activering network with one or more back-up ring networks for use during afailover condition. Back-up ring networks may share the same componentsor alternately incorporate one or more other components. For example, aLayer 2 CPE may have a first active ring network with a first set ofLayer 2 Aggregators which also provide a second logical standby ringnetwork for failover conditions. In other cases, the Layer 2 CPE mayhave a standby ring network which has different Layer 2 Aggregators,from its active ring network.

As previously described, the data link layer (Layer 2) is the protocollayer that transfers data between adjacent network nodes in a network.Specifically, the data link layer is concerned with local delivery ofdata between devices. Data link frame data does not cross the boundariesof a local network (and does not require network address resolution). Toclarify, network routing and global addressing are handled within thenetwork layer (Layer 3), whereas the data link layer protocols focus onlocal delivery (next “hop” delivery), and medium access control.Traditionally, data link layer delivery is based on unambiguousaddresses. For example, the frame header contains source and destinationaddresses that uniquely identify a source device and a destinationdevice. In contrast to the hierarchical and routable addresses of thenetwork layer, the data link layer addresses are “flat” i.e., no part ofthe address can be used to identify the logical or physical group towhich the address belongs on the LAN segment.

Various data link protocols may provide different levels of complexityand/or functionality. For example, certain data link protocols mayincorporate error checking/correction (e.g., bit error rate (BER), blockerror rate (BLER), packet error rate (PER), cyclic redundancy check(CRC), parity, forward error correction (FEC), checksum, etc.),acknowledgement/non-acknowledgment (ACK/NACK), flow control, etc.

Consider the following deployment: a plurality of Layer 2 CPEs issupported by two (2) Layer 2 Aggregator devices. Each one of theplurality of Layer 2 CPEs has a distinct logical ring network thatconsists of itself and the two (2) Layer 2 Aggregator devices. Thelogical ring network is tunneled (e.g., the aforementioned MPLStransport routers) at the data link layer (Layer 2) thus, from Layer 3and above (e.g., network and transport layers (transport controlprotocol/internet protocol (TCP/IP), etc.) the nodes of the ring networkare “directly” connected.

In one exemplary embodiment, the ring network is configured to supportat least a primary path (e.g., a primary EVC), and a secondary path(e.g., a secondary EVC), each path is further backed with a standby path(e.g., a primary standby EVC, and a secondary standby EVC). The ringnetwork includes ITU-T G.8032 switch mechanisms which are configured toautomatically switch between the active paths and the standby paths whena failure has been detected (e.g., where the ring is broken). Moreover,it should be appreciated that physical redundancy may provide yetanother layer of protection; for example, multiple physically redundantnetwork routers may be switched in to replace failing network routers,etc.

In slightly more detail, the ring network is established according tothe ITU-T G.8032 Ethernet Ring Protection Switching (ERPS). ERPS is oneexemplary implementation of Automatic Protection Switching (APS) at theservice VLAN level (not the port level); one path is blocked while theother remains active. In APS, each ring is a domain, which ischaracterized by a single “master node” and many “transit nodes”. Eachnode will have a primary port and a secondary port, both ports are ableto send control traffic to the master node; however, under normaloperation only the primary port on the master node is used (thesecondary port is blocked for all non-control traffic). When the ringfails, the devices that detect the failure send a control message to themaster node, and the master unblocks the secondary port and instructsthe nodes to flush their current transmit queues and reconfigure forsecondary port operation.

Referring back to the exemplary embodiment, the entire ring network isconstructed from “pseudo-wires” between each of the nodes of the ringnetwork. This “direct” connection provides multiple advantages. Firstly,failover mechanisms are greatly simplified. Since each node of the ringnetwork is logically directly connected, an error is immediately andunambiguously detected (e.g., a missed frame, etc.). For example, withinthe context of networks which utilize ITU-T G.8032 Ethernet ringprotection, the described architecture enables any of the components todetect and trigger failover recovery within 50 ms of a failure event.

Secondly, it should be appreciated that since each node of the ringnetwork can directly monitor network traffic and/or measure error rates,the overall network can be constructed from commodity components. Moredirectly, many manufacturers provide network diagnostic software whichmay not integrate properly with other manufacturer's software;integration problems have traditionally prevented multi-sourcing ofnetwork infrastructure. In contrast, the various embodiments describedherein enables each of the nodes to directly monitor/measure/diagnosetraffic based on the standardized Ethernet frame (based on e.g.,preamble, frame delimiter, MAC destination address, MAC source address,data payload, and frame check sequence, etc.)

Thirdly, those of ordinary skill in the related arts will readilyappreciate that these pseudo-wires behave as a direct logical connection(e.g., without higher level network routing, and/or unpredictablerouting delays). Thus, a device coupled to one end of a pseudo-wire isdirectly connected to a device coupled to the other end of thepseudo-wire. This logically direct linkage can greatly simplify timingcritical messaging. For example, a Layer 2 CPE device (such as acellular tower) with a pseudo-wire to the Layer 2 Aggregator of the CoreNetwork can rely on the stability offered by the pseudo-wire tofacilitate time sensitive messages e.g., IEEE 1588 timingsynchronization, etc. when coupled with existing queuing mechanisms thatprioritize timing frames based on the priority field (e.g., theaforementioned IEEE 802.1Q tag and/or MPLS EXP field).

In one exemplary embodiment, the Ethernet frames are tunneled viaMultiprotocol Label Switching (MPLS). MPLS provides a high speedtransport for variable length frames. The frames are routed according toone or more labels which may define various tiers of e.g., source,destination, etc. As the frames are routed from one MPLS router toanother, the labels may be replaced and rerouted. Since MPLS onlyrequires full network address resolution for connection establishment,routing can be performed at very high speeds and with minimal networkoverhead. Additionally, the variable length packets of MPLS can supportvirtually any data encapsulation; only the MPLS labels are modified intransit, the encapsulated data is not altered during transit.

At step 406 of the method 400, the deployed ring networks transact databetween one or more ingress points (e.g., the cellular tower, etc.) andone or more egress points (e.g., the MSP routers, etc.). In oneexemplary embodiment, each Layer 2 CPE services a cellular tower, andtwo (2) Layer 2 Aggregator devices are connected to the MSP routers. Inthis manner, the MSP routers can support multiple cellular towers (eachprotected with a distinct ring network in the backhaul providernetwork). By maintaining a distinct ring network for each cellulartower, the MSP routers can incrementally add, remove, and/or update eachof the cellular towers without adverse affect to the other networks.While the exemplary discussions herein are directed to a point-to-pointtype tunnel (e.g., between the cellular tower and the MSP) implementedwithin a ring network, it is appreciated that a ring topology mayreadily encompass any number of logical entities. For example, the ringmay support multiple cellular towers and/or MSP. One common example ofsuch a configuration is a so-called “daisy chain” of nodes, otherexamples include e.g., trees, hubs, etc.

Apparatus—

Referring now to FIGS. 5, 6, and 7, exemplary network components usefulin conjunction with the various methods described herein areillustrated.

Exemplary Consumer Premises Equipment (CPE)—

FIG. 5 is a block diagram illustrating an exemplary embodiment of aConsumer Premises Equipment (CPE) 500 for use in providing networkedoperation in conjunction with the generalized architecture of FIGS. 3Aand 3B. As shown, the Layer 2 CPE generally comprises a Layer 2 capablenetwork interface 502 configured to interface a backhaul network, aconsumer premises interface 504, a processor 508 and associated storage506 (discussed in greater detail below). While the term CPE is usedherein, it should be readily appreciated that the following discussionis broadly applicable to any “last mile” type device which is configuredto provide the final source and destination type forwarding of customerdata. Common examples of such last mile type devices include e.g.,cellular towers, gateways, consumer equipment, etc.

The Layer 2 capable network interface 502 provides, inter alia, contentand data delivery to and from a backhaul type network, such as theherein described Layer 2 based ring network, traditional legacynetworks, and/or hybrids thereof, etc. The premises interface 504provides inter alia, communication between the CPE 500 and variousdevices within the consumer premises, such as e.g., client mobiledevices, Internet Protocol (IP) enabled devices, gateways, etc. Forexample, the premises interface 504 may be used to connect to a cellularsite, base station (BS), home gateway, multi-home gateway, etc.

The processor 508 may include one or more of a digital signal processor,microprocessor, field-programmable gate array, or plurality ofprocessing components mounted on one or more substrates. The processingsubsystem 508 may also comprise an internal cache memory. The processingsubsystem is in communication with a memory subsystem 506, the latterincluding memory which may for example comprise SRAM, flash, and/orSDRAM components. The memory subsystem may implement one or more of DMAtype hardware, so as to facilitate data accesses as is well known in theart. The memory subsystem of the exemplary embodiment containscomputer-executable instructions which are executable by the processorsubsystem.

In the illustrated embodiment, the processor 508 is configured to run avirtual network application 510 thereon. The virtual network application510 is configured to: (i) receive premises traffic that is addressed toa network entity outside of the premises and package the traffic fortransmission via the Layer 2 capable network interface 502 (coupled to aLayer 2 ring network based backhaul); and (ii) receive Layer 2 ringnetwork based packets that encapsulate data and determine if theencapsulated data includes data that should be forwarded via thepremises network.

In one embodiment, the premises interface 504 is configured to transactone or more network address packets with other networked devicesaccording to a network protocol. As is commonly implemented within therelated arts, network addressing provides each node of a network with anaddress that is unique to that network; the address can be used tocommunicate (directly, or indirectly via a series of “hops”) with thecorresponding device. In more complex networks, sub-networks may be usedto assist in address exhaustion (e.g., one address is logically dividedinto another range of network addresses). Common examples of OpenSystems Interconnection (OSI) based network routing protocols includefor example: Internet Protocol (IP), Internetwork Packet Exchange (IPX),and OSI based network technologies (e.g., Asynchronous Transfer Mode(ATM), Synchronous Optical Networking (SONET), Synchronous DigitalHierarchy (SDH), Frame Relay, etc.)

In one embodiment, the Layer 2 capable network interface 502 isconfigured to transact one or more data link frames with other Layer 2capable devices according to a data link protocol. In some variants, theLayer 2 capable network interface 502 is additionally configured totransact one or more network address packets with other networkeddevices according to a network protocol (e.g., Layer 3 capabilities), tosupport management of the CPE 500.

In one exemplary embodiment, the exemplary Layer 2 CPE is configured toconnect to one or more other Layer 2 devices via a tunneled ringnetwork. The Layer 2 CPE is configured to transact data via the ringnetwork, perform failover switching, and/or measure and monitor datatraffic. Generally, the Layer 2 CPE is configured to connect accessnetworks (e.g., consumer or network operator equipment) to the backhaulnetwork. For example, in one exemplary embodiment, the Layer 2 CPE iscoupled to a cellular tower site. In other embodiments, the Layer 2 CPEprovides network connectivity for a small business premises and/orresidential premises.

Exemplary Layer 2 Aggregator Device—

FIG. 6 is a block diagram illustrating an exemplary embodiment of aLayer 2 Aggregator device 600 for use in providing networked operationin conjunction with the generalized architecture of FIGS. 3A and 3B. Asshown, the aggregator device 602 generally comprises a Layer 2 capablenetwork interface 602, a backbone interface 604 (also referred to as anExternal Network-Network Interface (ENNI)), a processor 608, and anassociated storage device 606 (described in greater detail below).

The Layer 2 capable network interface 602 provides, inter alia, contentand data delivery to and from a backhaul type network, such as theherein described Layer 2 based ring network, traditional legacynetworks, and/or hybrids thereof, etc. The backbone interface 604provides inter alia, communication between the Layer 2 Aggregator deviceand a destination network. In some cases, the destination network may bethe mobile service provider (MSP). In other implementations, the Layer 2network interface may provide access to the broader Internet backbone.Generally, it is appreciated that the Internet backbone refers to theprincipal data routes between large, strategically interconnectednetworks and core routers on the Internet hosted by e.g., commercial,government, academic and other high-capacity network centers, etc.

In some embodiments, the Layer 2 Aggregator device 600 may additionallyinclude a distinct ring interface (not shown) used to interconnect theL2 Aggregator device 600 to another Layer 2 Aggregator device 600 in theMobile Switching Center (MSC). In other embodiments, the ring interfacemay be implemented via the Layer 2 capable network interface 602, asecond Layer 2 capable network interface, or the backbone interface 604.

The processor 608 may include one or more of a digital signal processor,microprocessor, field-programmable gate array, or plurality ofprocessing components mounted on one or more substrates. The processingsubsystem 608 may also comprise an internal cache memory. The processingsubsystem is in communication with a memory subsystem 606, the latterincluding memory which may for example comprise SRAM, flash, and/orSDRAM components. The memory subsystem may implement one or a more ofDMA type hardware, so as to facilitate data accesses as is well known inthe art. The memory subsystem of the exemplary embodiment containscomputer-executable instructions which are executable by the processorsubsystem.

In the illustrated embodiment, the processor 608 is configured to run avirtual network application 610 thereon. The virtual network application610 is configured to: (i) receive traffic that is addressed to a networkentity within the premises associated with a CPE and package the trafficfor transmission via the Layer 2 capable network interface 602 (coupledto a Layer 2 ring network based backhaul); and (ii) receive Layer 2 ringnetwork based packets that encapsulate data and determine if theencapsulated data includes data that should be routed via the backboneinterface 604.

In one embodiment, the ENNI 604 is configured to transact one or morenetwork address packets with other networked devices according to anetwork protocol. In one exemplary embodiment, the backbone interface604 is directly coupled to the MSP's network routers.

In one embodiment, the Layer 2 capable network interface 602 isconfigured to transact one or more data link frames with other Layer 2capable devices according to a data link protocol. In some variants, theLayer 2 capable network interface 602 is additionally configured totransact one or more network address packets with other networkeddevices according to a network protocol (e.g., Layer 3 capabilities),where network capabilities are useful in hybrid deployments (e.g., wherethe backhaul may incorporate a combination of Layer 2 and Layer 3network components).

In one embodiment, each Layer 2 Aggregator device is configured toconnect to one or more other Layer 2 devices via a tunneled ringnetwork. The Layer 2 Aggregator is configured to transact data via thering network interface, perform failover switching, and/or measure andmonitor data traffic. Generally, the Layer 2 Aggregator is configured toconnect the backhaul network to the core network. For example, in oneexemplary embodiment, the Layer 2 Aggregator is coupled to one or moreMSP routers. The Layer 2 Aggregator further combines or “aggregates” theserviced Consumer Premises Equipment (CPE). In some embodiments, theLayer 2 Aggregator only services Layer 2 CPEs; alternatively, the Layer2 Aggregators can service a mixed population of both Layer 2 CPEs andlegacy (Layer 3) CPEs.

Exemplary Layer 2 Network Interface Device—

FIG. 7 is a block diagram illustrating an exemplary embodiment of aLayer 2 Network Interface device 700 for use in providing networkedoperation in conjunction with the generalized architecture of FIGS. 3Aand 3B. As shown, the network interface device 700 generally comprises afirst and second Layer 2 capable network interface 702A and 702B, aprocessor 708, and an associated storage device 706 (described ingreater detail below).

The Layer 2 capable network interface 702A and 702B provide, inter alia,content and data forwarding within a backhaul type network, such as theherein described Layer 2 based ring network, traditional legacynetworks, and/or hybrids thereof, etc. In one exemplary embodiment, theLayer 2 data protocol comprises an IEEE 802.3 Ethernet protocol.

The processor 708 may include one or more of a digital signal processor,microprocessor, field-programmable gate array, or plurality ofprocessing components mounted on one or more substrates. The processingsubsystem 708 may also comprise an internal cache memory. The processingsubsystem is in communication with a memory subsystem 706, the latterincluding memory which may for example comprise SRAM, flash, and/orSDRAM components. The memory subsystem may implement one or a more ofDMA type hardware, so as to facilitate data accesses as is well known inthe art. The memory subsystem of the exemplary embodiment containscomputer-executable instructions which are executable by the processorsubsystem.

In the illustrated embodiment, the processor 708 is configured to run adata link layer ring network application 710 thereon. The ring networkapplication 710 is configured to: (i) receive traffic that includes aVLAN tag associated with the device 700, and encapsulated data; and (ii)add a VLAN tag associated with a neighbor node of the ring network andforward the traffic to the neighbor node.

In one embodiment, the Layer 2 capable network interfaces 702A and 702Bare configured to transact one or more data link frames with other Layer2 capable devices according to a data link protocol. In some “hybrid”variants, the network interfaces 702A and 702B are additionallyconfigured to transact one or more network address packets with othernetworked devices according to a network protocol (e.g., Layer 3capabilities), where network capabilities are useful in hybriddeployments (e.g., where the backhaul may incorporate a combination ofLayer 2 and Layer 3 network components).

Example Operation

Referring now to FIG. 8, one exemplary method 800 for implementing anITU-T G.8032 ring network within a backhaul distribution network withMPLS capability is illustrated.

At step 802 of the method 800, a backhaul provider upgrades itsdistribution networks to MPLS network routers; throughout the upgraderollout, the distribution network routes data according to fixed LDPtunnels to assure appropriate QoS.

At step 804 of the method 800, the backhaul provider upgradesappropriate endpoints with Layer 2 aggregation devices and/or Layer 2CPE based on a determined ring network configuration. Determination ofupgrade priority may be based on e.g., bandwidth requirements, monetaryconsideration, network congestion, network planning, etc. For example,in one exemplary embodiment, the backhaul provider may opt to upgradethe endpoints associated with a MSP cellular towers first.

At step 806 of the method 800, enable an ITU-T G.8032 ring network forthe appropriate endpoints. The ring network transfers throughout thering without requiring higher level network address resolution.

At step 808 of the method 800, as additional nodes are equipped, thering network can be expanded to incorporate new nodes. For example, thebackhaul provider may add equipment based on Internet Service Provider(ISP) network traffic.

It will be recognized that while certain aspects of the disclosure aredescribed in terms of a specific sequence of steps of a method, thesedescriptions are only illustrative of the broader methods describedherein, and may be modified as required by the particular application.Certain steps may be rendered unnecessary or optional under certaincircumstances. Additionally, certain steps or functionality may be addedto the disclosed embodiments, or the order of performance of two or moresteps permuted. All such variations are considered to be encompassedwithin the embodiments disclosed and claimed herein.

While the above detailed description has shown, described, and pointedout novel features of the disclosed embodiments as applied to varioussystems, it will be understood that various omissions, substitutions,and changes in the form and details of the device or process illustratedmay be made by those skilled in the art without departing from theprinciples described herein. The foregoing description is of the bestmode presently contemplated. This description is in no way meant to belimiting, but rather should be taken as illustrative of the generalprinciples of the disclosure. The scope of the disclosure should bedetermined with reference to the claims.

What is claimed is:
 1. A method for operating a content distributionnetwork, said method comprising: enabling at least one user interface tocommunicate with an edge network, said at least one user interfacecomprising a Multiprotocol Label Switching (MPLS) compliant interface;receiving at least one data frame from a first other node of a backhaulnetwork, said backhaul network comprising at least a ITU-T G.8032compliant logical ring network; transmitting said at least one dataframe to a second other node of said ITU-T G.8032 compliant logical ringnetwork; determining when said at least one data frame comprises atleast one packet for said edge network; and routing said at least onepacket via said edge network; wherein said edge network comprises acellular tower site.
 2. A non-transitory computer-readable apparatuscomprising media configured to store a computer program thereon, saidcomputer program comprising a plurality of instructions which areconfigured to, when executed by a processor of a network apparatus:receive via an interface of the network apparatus at least one dataframe originated from a first node of a backhaul logical ring network,the logical ring network being ITU-T G.8032 compliant; insert into saiddata frame a second label associated with a second node of said logicalring network in place of a first label associated with a network router;transmit said at least one data frame to a second node of said backhaullogical ring network; determine that said at least one data framecomprises at least one packet for a cellular site associated with anedge network; and route at least said at least one packet via said edgenetwork to said cellular site.
 3. The method of claim 1, wherein saidlogical ring network services a combination of retail and carrierEthernet applications characterized by distinct service level agreements(SLAs).
 4. The method of claim 1, wherein said logical ring networkcomprises at least a first active path in a primary ring, and a secondactive path in a secondary ring, and said at least one data frame istransacted via said first active path in the primary ring and saidsecond active path in the secondary ring.
 5. The method of claim 4,wherein: said logical ring network further comprises at least a thirdpath in a standby primary ring and a fourth path in a standby secondaryring, said third path and said fourth path being selectively blocked;and upon detection of a ring failure, at least one of said blocked thirdpath and said fourth path are unblocked, and said at least one dataframe is transacted therethrough.
 6. The method of claim 1, wherein saidrouting occurs according to one or more fixed Label DistributionProtocol (LDP) tunnels.
 7. The method of claim 1, wherein saiddetermining is based at least in part on a destination Internet Protocol(IP) address found in said at least one packet.
 8. The method of claim1, wherein said at least one data frame comprises encapsulated data anda stack comprising: (i) a first stack layer configured to providerouting information; (ii) a second stack layer configured to specify atransport network service endpoint; (iii) a third stack layer configuredto identify an appropriate private network for said encapsulated data;and wherein each of said first, second, and third stack layers areassociated with corresponding quality of service (QoS) informationuseful for prioritization within said associated layer.
 9. Thenon-transitory computer-readable apparatus of claim 2, wherein saidlogical ring network is configured to route said at least one data framebased on associated labels.
 10. The non-transitory computer-readableapparatus of claim 2, wherein said at least one data frame comprisesencapsulated data and a stack comprising: (i) a first stack layerconfigured to provide routing information; (ii) a second stack layerconfigured to specify a transport network service endpoint; (iii) athird stack layer configured to identify an appropriate private networkfor said encapsulated data; and wherein each of said first, second, andthird stack layers are associated with corresponding quality of service(QoS) information useful for prioritization within said associatedlayer.
 11. The non-transitory computer-readable apparatus of claim 2,wherein said logical ring network services a combination of retail andcarrier Ethernet applications characterized by distinct service levelagreements (SLAs).
 12. The non-transitory computer-readable apparatus ofclaim 2, wherein said logical ring network comprises at least a firstactive path in a primary ring and a second active path in a secondaryring, and said at least one data frame is transacted via said firstactive path in the primary ring and said second active path in thesecondary ring.
 13. The non-transitory computer-readable apparatus ofclaim 12, wherein: said logical ring network further comprises at leasta third path in a standby primary ring and a fourth path in a standbysecondary ring, said third path and said fourth path being selectivelyblocked; and based at least on detection of a ring failure, at least oneof said third path and said fourth path are unblocked, and said at leastone data frame is transacted therethrough.
 14. The non-transitorycomputer-readable apparatus of claim 2, wherein said determination isbased at least in part on a destination Internet Protocol (IP) addressfound in said at least one packet.
 15. An apparatus, comprising: a firstnetwork interface configured to communicate with a backhaul networkcomprising at least a logical ring network, said first network interfacecomprising a Multiprotocol Label Switching (MPLS) compliant interface;an interface configured to communicate with an edge network; aprocessor; and a non-transitory computer readable medium comprising atleast one computer program configured to, when executed by saidprocessor, cause said apparatus to: receive at least one data frame froma first other node of said logical ring network, the logical ringnetwork being ITU-T G.8032 compliant; transmit said at least one dataframe to a second other node of said logical ring network; determinewhen said at least one data frame comprises at least one packet for acellular site associated with said edge network; and route said at leastone packet via said edge network to said cellular site.
 16. Theapparatus of claim 15, wherein said first network interface is furtherconfigured to communicate with a Virtual Local Area Network (VLAN),tunneled via said backhaul network.
 17. The apparatus of claim 16,wherein said VLAN comprises a first network protocol within a payload ofa second network protocol.
 18. The apparatus of claim 16, wherein saidat least one packet comprises a VLAN packet.
 19. The apparatus of claim18, wherein a second label associated with said second other node ofsaid logical ring network replaces a first label associated to said atleast one VLAN packet.
 20. The apparatus of claim 15, wherein saidlogical ring network services a combination of Retail and CarrierEthernet applications characterized by distinct Service Level Agreements(SLAs).